Playback of interactive television applications

ABSTRACT

A method and apparatus for controlling the playback of interactive television applications associated with a recorded program when a viewer is using trick play modes of operation is disclosed. When a viewer activates a fast forward operation. All events that the application would normally receive are supplied to the application at a faster rate than normal. When the viewer activates a rewind operation, the application is either paused until the playback of the rewound program returns to the location where the rewind operation began, or the application is loaded with a previous saved valid state for the location to which the program is rewound.

The invention relates to the playback of interactive televisionapplications, and more particularly to the operation of interactivetelevision applications during trick play modes.

Interactive television systems enable television sets to be used toprovide various new means for providing services to viewers. Interactivetelevision systems are capable of displaying text and graphic images inaddition to typical video program streams. Interactive televisionsystems are also capable of registering viewer actions or responses.Features of interactive television include a variety of marketing,entertainment and educational capabilities such as allowing a user tointeract with televised programs by ordering advertised products orservices, competing against contestants in a game show, or requestingspecialized information regarding particular programs.

Typically, a broadcast service provider generates an interactivetelevision signal for transmission to a viewer's television. Theinteractive television signal includes an interactive portion consistingof application code and/or control information (events), as well as anaudio-video portion consisting of a television program. The broadcastservice provider combines the audio-video and interactive portions intoa single signal for transmission to a receiver connected to the user'stelevision. The signal is generally compressed prior to transmission andtransmitted through typical broadcast channels, such as cable televisionlines or direct satellite transmission systems.

The interactive functionality of the television is controlled by aset-top box connected to the television. The set-top box receives thesignal transmitted by the broadcast service provider, separates theinteractive portion from the audio-video portion and decompresses therespective portions of the signal. The set-top box uses the interactiveinformation to, for example, execute an application while theaudio-video information is transmitted to the television. The set-topbox may combine the audio-video with interactive graphics or audiogenerated by the interactive application prior to transmitting theinformation to the television. The interactive graphics and audio maypresent additional information to the viewer or may prompt the viewerfor input. The set-top box may provide viewer input or other informationto the broadcast service provider via a modem connection.

The set-top box may include or be attached to a video recorder andplayback system such as a DVD player or a hardware device driver (HDD).Television programs can be recorded on discs or another medium forplayback at a later time. In addition, DVD discs can be purchased withvideo programs and information for running interactive televisionapplications already stored on the discs. When running an applicationthat is retrieved directly from a live broadcast, the application timeincreases linearly and a viewer can not jump back and forth in theprogram using trick play modes., i.e., fact forward, jump forward,rewind, jump back, etc. However, when replaying a recorded program, theviewer can fast forward or rewind the recorded program to move forwardor back in the recorded program. However, problems may occur when theviewer selects these trick play modes. For example, it is possible thatan interactive television application that was written for linearplayback will not be able to handle the fast forward and fast rewindoperations, which are non-linear operations. Reasons for this includethat an application may receive events that are broadcast for theapplication, and the nonlinear playback could result in missing suchevents or receiving such events multiple times.

One way to solve this problem with non-linear operations is to stop andstart the application when the trick play modes are used. However, thissolution may be very disturbing to the viewer. For example, if theapplication is associated with a television program containingcommercials and the viewer skips over the commercials, the applicationwill be stopped sued then restarted. Furthermore, applications maymalfunction if events are not fed to the application during the fastforward operation.

SUMMARY OF THE INVENTION

It is an object of the invention to overcome the problems with priorsystems by providing a method and apparatus for improved behavior ofinteractive television applications and improved viewer interactionduring and after trick play mode operation. According to one embodimentof the invention, a method and apparatus for controlling an interactivetelevision application during a fast forward operation of an associatedrecorded program is disclosed. When a fast forward request is detected,the speed at which the recorded program is provided to a display isincreased. The display of information from the application is thenblocked during the fast forward operation. Events for the applicationare detected in the recorded program and supplied to the application forprocessing during the fast forward operation. By providing the events tothe application during the fast forward operation, the application willbe in a consistent state at the end of the fast forward operation.

According to another embodiment of the invention, a method and apparatusfor controlling an interactive television application during a rewindoperation of an associated recorded program is disclosed. When a rewindrequest is detected, the application is paused. The location of therecorded program when the rewind operation is started is then marked.The application is kept paused until the recorded program has beenreplayed to the marked location, wherein the application is restarted.By pausing the application during the rewind operation, duplicate eventsare not supplied to the application, thus ensuring that the applicationis in a consistent state.

According to another embodiment of the invention, a method and apparatusfor controlling an interactive television application using stored statehistories is disclosed. While a recorded program is being processed fordisplay, a state of the application is determined at predetermined timesduring the display of the recorded program. Each state is then stored ina memory with an identifier. The location of the recorded program isdetermined at end of a rewind operation. A stored state from the memorywhich is valid for the determined location as identified by theidentifier stored with the state is then selected and loaded into theapplication. By dynamically storing the state history of an application,a viewer is given the opportunity to interact with replayed portions ofthe recorded program.

These and other aspects of the invention will be apparent from andelucidated with reference to the embodiments described hereafter.

BRIEF DESCRIPTION OF THE DRAWINGS

The invention will now be described, by way of example, with referenceto the accompanying drawings, wherein:

FIG. 1 is a block schematic representation of an audio-videoinstallation according to one embodiment of the invention;

FIG. 2 is a block schematic representation of a set-top box according toone embodiment of the invention;

FIG. 3 is a flow chart depicting the operation of the set-top box when afast forward operation has been selected according to one embodiment ofthe invention;

FIG. 4 is a flow chart depicting the operation of the set-top box when arewind operation has been selected according to one embodiment of theinvention;

FIG. 5 is a flow chart depicting the operation of the set-top box when acombined rewind and fast forward operation has been selected accordingto one embodiment of the invention; and

FIG. 6 is a flow chart depicting the operation of the set-top box when arewind operation has been selected according to one embodiment of theinvention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

FIG. 1 illustrates and audio-video apparatus suitable to host theinvention. The apparatus comprises un input terminal 1 for receiving adigital video signal to be recorded on a disc 3. Further, the apparatuscomprises an output terminal 2 for supplying a digital video signalreproduced from the disc. These terminals may in use be connected via adigital interface to a digital television receiver and decoder 12 in theform of a set-top box (STB), which also receives broadcast signals fromsatellite, cable or the like, in MPEG TS format. The decoder 12 providesdisplay signals to a display device 14, which may be a conventionaltelevision set.

The video recording apparatus as shown in FIG. 1 is composed of twomajor system parts, namely the disc subsystem 6 and the video recordersubsystem 8, controlling both recording and playback. The two subsystemshave a number of features, as will be readily understood, including thatthe disc subsystem can be addressed transparently in terms of logicaladdresses (LA) and can guarantee a maximum sustainable bit-rate forreading and/or writing data from/to the disc.

Suitable hardware arrangements for implementing such an apparatus areknown to one skilled in the art, with one example illustrated in patentapplication WO-A-00/00981. The apparatus generally comprises signalprocessing units, a read/write unit including a read/write headconfigured for reading from/writing to disc 3. Actuators position thehead in a radial direction across the disc, while a motor rotates thedisc. A microprocessor is present for controlling all the circuits in aknown manner.

Referring to FIG. 2, a block diagram of a set-top box 12 is shown. Abroadcast signal is received and fed into a tuner 31. The timer 31selects the channel on which the broadcast audio-video-interactivesignal is transmitted and passes the signal to a processing unit 32. Theprocessing unit 32 demultiplexes the packets from the broadcast signalif necessary and reconstructs the television programs and/or interactiveapplications embodied in the signal. The programs and applications arethen decompressed by a decompression unit 33. The audio and videoinformation associated with the television programs embodied in thesignal is then conveyed to a display unit 34, which may perform furtherprocessing and conversion of the information into a suitable televisionformat, such as NTSC or HDTV audio/video. Applications reconstructedfrom the broadcast signal are routed to random access memory (RAM) 37and are executed by a control system 35.

The control system 35 may include a microprocessor, micro-controller,digital signal processor (DSP), or some other type of softwareinstruction processing device. The RAM 37 may include memory units whichare static (e.g. SRAM), dynamic (e.g. DRAM), volatile or non-volatile(e.g., FLASH), as required to support the functions of the set-top box.When power is applied to the set-top box, the control system 35 executesoperating system code which is stored in ROM 36. The operating systemcode executes continuously while the set-top box is powered in the samemanner as the operating system code of a typical personal computer andenables the set-top box to act on control information and executeinteractive and other applications. The set-top box also includes amodem 38. The modem 38 provides both a return path by which viewer datacan be transmitted to the broadcast station and an alternate path bywhich the broadcast station can transmit data to the set-top box.

Although the term “set-top box” is used herein, it will be understoodthat this term refers to any receiver or processing unit for receivingand processing a transmitted signal and conveying the processed signalto a television or other monitor. The set-top box may be in a housingwhich physically sits on top of a television, it may be in some otherlocation to the television, or it may be incorporated into thetelevision itself.

As noted above, a viewer can use trick play modes of operation whenviewing recorded programs. The operation of the set-top box during thesevarious trick play modes according to different embodiments of theinvention will now be described.

According to one embodiment of the invention, moving forward in time(fast forward, jumping forward, etc.) is handled by leaving theapplication running and supplying events to the application as theyoccur during the fast forwarding of the video program. In essence, theapplication is run at a faster rate since the events are fed to theapplication at a faster rate than during normal play. By supplying theevents to the application during the fast forward operation, theapplication will be in the same state as if the program had been playedwithout non-linear time (no fast forward), when the viewer switches thevideo program back to a normal play mode. Thus, the application will bein a consistent state. It will be understood that some events may needto be queued if previous events are still being processed, but this willnot be noticeable to the viewer in most cases.

According to one embodiment of the invention, the application isprevented from displaying any information or images during the fastforward operation. While the following description refers to a fastforward operation, it will be understood that the invention also appliesto a jumping forward operation and any other similar forwardingoperation. If the application would normally expect viewer input fromevents which occur during the fast forward operation, the applicationwill simply use default inputs and continue on since the user is notgiven the opportunity to enter any information during the fast forwardoperation. Under normal operation there may be no user watching theprogram so the interactive application must behave correctly in the casewhere the user does not input any information. Therefore, there is somewell defined default behavior in the absence of user input both duringnormal play or fast forward operations.

FIG. 3 illustrates the operation of the set-top box during the fastforward operation. The tuner 31 receives recorded information from thevideo recorder subsystem 8. The processing unit 32 then processes therecorded information in step 302. The processing unit 32 sends theaudio-video information onto the decompression unit 33 for display onthe display 14. The processing unit also detects any events stored inthe recorded information and provides these events to the decompressionunit 33 and then to the control system 35 for processing with by theapplication in step 304. When the set-top box detects a fast forwardrequest from a viewer in step 306, the control system 35 blocks thedisplay of any information or images from the application in step 308.The set-top box then increases the speed at which the recorded programis read in a known manner to provide the requested fast forwardoperation in step) 310. During the fast forward operation, theprocessing unit 32 still detects the events associated with the fastforwarded recorded information and provides the events to the controlsystem 35 for processing by the application in step 312. If it isdetermined that the application is processing a prior event in step 314,the event(s) can be queued until the application is ready for the nextevent in step 316. When the set-top box 12 detects a normal play requesti.e., the end of fast forward operation, in step 318, the control system35 removes the block on the display of information or images from theapplication in step 320 and the set-top box and application return tonormal operation. After returning to normal play, the system willcontinue processing any events that are still queued. This process isthen repeated anytime a fast forward request is received during theviewing of the recorded information.

According to another embodiment of the invention, events can be storedoutside of the AN multiplexed data stream. By storing the events outsideof the A/V multiplexed data stream, it is easier to feed the events at afaster rate to the application since the complete AN multiplexed datastream does not have to be read to extract the events during the fastforward operation.

As will be explained in more detail below, one embodiment of theinvention pauses the operation of an application when the viewer selectsa skip back, rewind, or fast rewind operation and the applicationremains paused during playback of the portion of the video program thathave already been viewed. All events that occur during the replayedsection of the video program are duplicates so they are not provided tothe paused application. When the replayed section of video programreaches the point where the viewer started the rewind operation, theapplication resumes normal operation and subsequent events are fed tothe application.

FIG. 4 illustrates the operation of the set-top box during a rewindoperation. While the following description refers to a rewind operation,it will be understood that the invention applies to skip back, fastrewind and other similar rewinding operations. The tuner 31 receivesrecorded information from the video recorder subsystem 8. The processingunit 32 then processes the recorded information in step 402. Theprocessing unit 32 sends the audio-video information onto thedecompression unit 33 for display on the display 14. The processing unitalso detects any events stored in the recorded information and providesthese events to the decompression unit 33 and then to the control system35 for processing with by the application in step 404. When the set-topbox detects a rewind request from a viewer in step 306, the controlsystem 35 pauses the application in step 408. For example, theapplication manager can use a pause application program interface (API)to pause the application during the rewind and subsequent replay of thealready viewed portion of the video program, but the invention is notlimited thereto. The control system 35 also marks the point in the videoprogram where the rewind operation was requested in step 410. Themarking of the location of the video can be performed in a variety ofknown ways, for example, by noting the elapsed time of the videoprogram. When the set-top box detects the end of the rewind request instep 412, the set-top box begins normal processing of the video programexcept that detected events are not supplied to the paused applicationin step 414 and the video program is displayed in step 416. When thecontrol system 35 detects that the video program has reached the markedlocation where the rewind request was received in step 418, the controlsystem resumes the normal operation of the application in step 420. Forexample, the application manager can use a start API to resume thenormal operation of the application, but the invention is not limitedthereto. This process is then repeated anytime the set-top box receivesa rewind request.

It is also possible for a viewer to rewind and then fast forward thevideo past a point where normal play was stopped. In this situation, theapplication is paused during the rewind and fast forward operation untilthe point where normal play was interrupted. Then, the applicationresumes operation and receives any detected events during the fastforward operation from the point where the normal play was suspended.

FIG. 5 is a flow chart illustrating the operation of a set-top boxduring a combined rewind/fast forward operation according to oneembodiment of the invention. When a rewind request is detected, theset-top box operates in accordance with steps 402-412 of FIG. 4. Whenthe rewind operation is stopped, the video program resumes normal playand the application remains paused in step 504. When a fast forwardrequest is detected in step 506, the fast forward operation is begin inthe manner described above with respect to FIG. 3. However, theapplication remains paused and events are not supplied to theapplication until the video program reaches the location where therewind operation was requested in step 508. When the video programreaches the point where the rewind request was received in step 510, thecontrol system 35 resumes normal operation of the application and eventsare supplied to the application as they are detected during the fastforward operation in step 512. The control system, however, blocks thedisplay of any information or images from the application until the endof the fast forward operation as described above with respect to FIG. 3.Once the fast forward operation ends, the block on the display ofinformation or images from the application is removed in steps 514-516.

In some situations, it may be desirable to give a viewer the opportunityto interact with the application after the video stream has been rewoundand begins normal play. For example, if the application is a game, itmight be desirable to allow the user to rewind the video stream andreplace an incorrect answer with a correct answer. According to oneembodiment of the invention, this capability can be achieved bydynamically storing a history of states of the application during normalplay of the video program so that when the viewer rewinds the videoprogram, the application can revert back to the proper state. Forexample, the state of the application for different sections of thevideo program can be stored while the video stream is being viewed. Thedifferent sections can be defined or bounded by changes in the state ofthe application or can be determined and stored at predeterminedintervals of time, but the invention is not limited thereto. The amountof data required to store the state of the application will depend onthe complexity of the application. For very complex applications withconstantly changing states, it may be impractical to store a completehistory of states, but rather a partial history only including recentchanges could be stored. Alternatively, the predetermined intervalbetween when a state is stored can be increased so that less states arestored in the memory.

FIG. 6 illustrates the operation of the set-top box implementing theembodiment where the states of an application are stored in memory. Instep 602, the set top box receives recorded information from therecorder an processes the recorded information in the manner describedabove with respect to FIGS. 3 and 4. While the recorded information isbeing processed and the resulting video program is being displayed tothe viewer, the control system 35 periodically determines and stores thestate of the application in steps 604 and 606. The control system 35 maystore the state each time the application changes state or atpredetermined intervals of time. For example, the control system maystore the state of the application every minute or every 5 minutes, butthe invention is not limited thereto. Along with the state information,the control system 35 also stores information which defines theboundaries for which each stored state is valid. For example, the timeperiod covered by each state can be stored to identify when each stateis valid. The set-top box continues this operation until a rewindrequest is detected in step 608. The set-top box then determines thelocation of the video program at the end of the rewind operation in step610. The control system then loads the appropriate valid state frommemory into the application in step 612. One way to determine theappropriate valid state of the application is to base all of the storedinformation on a clock or timer in the set-top box. For example, thetimer can be set to zero at the start of the video program. The timerthen records the elapsed time as the video program is displayed. Theboundaries of the stored states can then be defined by the time on thetimer. As a result, if the viewer rewinds the video program back to atimer setting of 1:12:33, the control system will select the state ofthe application which includes this time within it's boundaries.Alternatively, timestamps stored as part of the A/V data stream can alsobe used. It will be understood that other methods can also be used tomark the boundaries of each stored state and the invention is notlimited to this example. Once the appropriate state has been loaded intothe application, the set-top box resumes normal play operation in step614.

The above-described embodiments ensure that an interactive applicationsees linear time even if the viewer requests non-linear operations andno unexpected behavior occurs and the application is always in aconsistent state. Thus, all applications will behave in a consistent wayeven though the application may not have been written for non-linearoperations. Furthermore, the behavior of the applications is predictableand logical from the point of view of the viewer. It will be understoodthat the different embodiments of the invention are not limited to theexact order of the above-described steps as the timing of some of thesteps can be interchanged without affecting the overall operation of theinvention.

1. A method for controlling an interactive television application duringa trick play mode operation of an associated recorded program,comprising the steps of: detecting a trick play mode request;determining the type of trick play mode requested; when the trick playmode is determined to be a fast forward operation, increasing speed atwhich the recorded program is provided to a display (14); blockingdisplay of information from said application; detecting events for saidapplication in said recorded program; and supplying detected events tosaid application for processing during the fast forward operation. 2.The method for controlling an interactive television applicationaccording to claim 1, wherein the fast forward operation is a fastforward or jump forward operation.
 3. The method for controlling aninteractive television application according to claim 1, furthercomprising the step of: queuing detected events for said application ifthe application is processing a prior event.
 4. The method forcontrolling an interactive television application according to claim 1,further comprising the steps of: when the trick play mode is determinedto be a rewind operation, pausing the application; marking the locationof the recorded program when the rewind operation is started; keepingthe application paused until the recorded program has been replayed tothe marked location; and restarting the application when the recordedprogram reaches the marked location.
 5. The method for controlling aninteractive television application according to claim 4, wherein therewind operation is a rewind, fast rewind or jump back operation.
 6. Themethod for controlling an interactive television application accordingto claim 4, further comprising the steps of: detecting a fast forwardrequest before the recorded program has reached the marked location;increasing the speed of the display of the recorded program; supplyingdetected events to the application dining the fast forward operationafter the recorded program has reached the marked location; and blockingdisplay of information from the application during the fast forwardoperation.
 7. A method for controlling an interactive televisionapplication associated with a recorded program, comprising the steps of:processing the recorded program for display; determining a state of theapplication at predetermined times during the display of the recordedprogram; storing each state in a memory with an identifier; determininglocation of recorded program at end of a rewind operation; selecting astored state from the memory (36) which is valid for the determinedlocation as identified by the identifier stored with the state; andloading the selected state into the application.
 8. The method forcontrolling an interactive television application according to claim 7,wherein the predetermined times are each time the application changesstate.
 9. The method for controlling an interactive televisionapplication according to claim 7, wherein the predetermined times arepredetermined intervals of time.
 10. The method for controlling aninteractive television application according to claim 7, wherein theidentifier identifies the boundaries of the validity of the state. 11.The method for controlling an interactive television applicationaccording to claim 7, wherein the identifier identifies the portion ofthe recorded program for which the state is valid.
 12. An apparatus forcontrolling an interactive television application during a fast forwardoperation of an associated recorded program by a method as claimed inclaim
 1. 13. An apparatus for controlling an interactive televisionapplication during a rewind operation of an associated recorded programby a method as claimed in claim
 4. 14. An apparatus for controlling aninteractive television application associated with a recorded program bya method as claimed in claim
 7. 15. An apparatus for controlling aninteractive television application during a trick play mode operation ofan associated recorded program, comprising: means for detecting (12) atrick play mode request and determining the type of trick play moderequested; when the trick play mode is determined to be a fast forwardoperation, means for increasing speed (12) at which the recorded programis provided to a display (14); a processing unit (32) for detectingevents for said application in said recorded program; and a controlsystem (35) for blocking display of information from the application andfor supplying detected events to the application for processing duringthe fast forward operation.
 16. The apparatus for controlling aninteractive television application according to claim 15, wherein thefast forward operation is a fast forward or a jump forward operation.17. The apparatus for controlling an interactive television applicationaccording to claim 15, wherein the control system (35) queues detectedevents for said application in a memory (36) if the application isprocessing a prior event.
 18. The apparatus for controlling aninteractive television application according to claim 15, wherein whenthe trick play mode request is determined to be a rewind request, thecontrol system (35) pauses the application in response to the rewindrequest, wherein the control system marks the location of the recordedprogram when the rewind operation is started, keeps the applicationpaused until the recorded program has been replayed to the markedlocation and restarts the application when the recorded program reachesthe marked location.
 19. An apparatus for controlling an interactivetelevision application according to claim 18, wherein the rewindoperation is a rewind, fast rewind or jump back operation.
 20. Theapparatus for controlling an interactive television applicationaccording to claim 18, further comprising: means for detecting (12) afast forward request before the recorded program has reached the markedlocation and increasing the speed of the display of the recordedprogram; wherein the control system (35) supplies detected events to theapplication during the fast forward operation after the recorded programhas reached the marked location and blocks display of information fromthe application during the fast forward operation.
 21. An apparatus forcontrolling an interactive television application associated with arecorded program, comprising: a processing unit (32) for processing therecorded program for display; a control system (35) for determining astate of the application at predetermined times during the display ofthe recorded program; a memory (36) for storing each state with anidentifier; wherein when the processing unit (32) determines thelocation of recorded program at end of a rewind operation, the controlsystem (35) selects a stored state from the memory which is valid forthe determined location as identified by the identifier stored with thestate and loads the selected state into the application.
 22. Theapparatus for controlling an interactive television applicationaccording to claim 21, wherein the predetermined times are each time theapplication changes state.
 23. The apparatus for controlling aninteractive television application according to claim 21, wherein thepredetermined times are predetermined intervals of time.
 24. Theapparatus for controlling an interactive television applicationaccording to claim 21, wherein the identifier identifies the boundariesof the validity of the state.
 25. The apparatus for controlling aninteractive television application according to claim 21, wherein theidentifier identifies the portion of the recorded program for which thestate is valid.